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In this report data are presented that challenge the 
difficulty ordering for anaphoric syntax (e. g., pronouns) proposed 
by Bormuth, Manning, Carr, and Pearson in 1970. It is suggested that 
any such difficulty ordering resulting from tests of the form 
proposed by Bormuth (1970) will have uncontrolled variability due to 
semantic factors that have yet to be carefully analyzed and 
controlled. (Author) 
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Abstract 



Data ^re presented that challenge the difficulty ordering for ana- 
phoric syntax (e. g. , pronouns) proposed by Bormuth, Manning, Carr, 
and Pearson (1970), It is suggested that any such difficulty ordering 
resulting from tests of the form proposed by Bormuth (1970) will have 
uncontrolled variability due to semantic factors that have yet to be care- 
fully analyzed and controlled. 
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VARIABILITY IN CHILDREN'S COMPREHENSION 
OV SYNTACTIC STRUCTURES^ 

Alan M, Lesgold 
University of Pittsburgh 

Borniuth, Manning, Carr, and Pearson (1970) have reported re- 
sults suggesting that stable difficulty orderings could be obtained for 
three classes of syntactic forms containing a total of 55 separate var- 
iants. There wore difficulty differences both between and within the 

2 

three classes (int ra- sentence, inter-sentence, and anaphora ), from 
which they argued that these categories may be related to stages in a 
learning hierarchy for syntax comprehension skills. Further, there 
have been recent suggestions that additional experimentation of this 
sort, combined with linguistic analysis, will result in discovery of such 
a hierarchy (e. g. , Carroll, 1972; Frase, 1972). The present results, 
a replication of part of the Bormuth et al. work, suggest that this ap- 
proach is unlikely to succeed unless such factors as semantics and con- 
straints on information processing capacity are concurrently considered. 

The procedure Bormuth et al. used in determining the difficulty 
of the various syntax forms was simple and, at first p:lance, straight- 
forward. For a given anaphora form, for example, one sentence was 



The experiment was conducted by Hildrene DeGood. Karen 
Block and Charles Perfetti read an earlier draft and helped to improve 
it. 
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Anaphora is the term used to denote a struct ire in a sentence, 
e. g. , a pronoun, that derives its meaning from a previous sentence or 
an earlier part of the present sentence. 
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written and then a second was constructed with sonie reference back to 
the first (e. g. , John went to the store . He bought a pear . ). A para- 
graph was then constructed around the two-sentence cluster. A question 
was generated for the paragraph by substituting a wh - wo r d (Bormuth, 
1970) for the anaphora (e.g., Who bought a pear ?). Finally (for anaphora 
only), multiple-choice alternative answers were written. The item thus 
consisted of a paragraph, a question, and alternative answers to choose 
from. 

There is a basic probleni in using procedures of this sort for 
measuring the difficulty of one syntactic structure relative to another. 
This is the confounding of syntax with semantics. Consider a potential 
comparison between a personal pronoun structure (e.g. , lie) and a pro- 
clause form (in which that or £o might stand for an entire clause). The 
two syntactic forms occur indifferent semantic contexts: There is no 
syntactic transformation series that will produce a personal pronoun 
from the semantic structure underlying a pro-clause. Now this does 
not necessarily mean that a difficulty ordering such as that of Bormuth 
et a] . should be discounted. After all, the confounding of syntax and 
semantics may, for such purposes, be complete. 

If the Bormuth et al. data are consistently replicable, then we 
can still look for a learning hierarchy related to that data, even though 
we would not know, initially, the relative roles of syntax and semantics. 
On the other hand, it may be that the difficulty ordering is not stable, 
that the particular confoundings of syntax and semantics in the item 
forms of Bormuth et al. are only partial. Then, if semantic content 
is not controlled, spurious measurements of syntax processing ability 
may result. If the latter is the case, then a difficulty ordering is pre- 
mature until a semantic analysis as complete as Bormuth's (1970) 



syntactic analysis is available. The present results arc a demonstra- 
tion that the difficulty ordering of syntactic structures is not stablo, im- 
plying that such further analyses are necessary. 

There is another problem in measuring syntax difficulty. Some- 
times the answer to a question may be betrayed by semantic constraints. 
The paragraph may contain only one semantically possible answer to the 
question. For example, a who question after a paragraph ^^ith only one 
anin^ate noun could be answered without the use of any skills related to 
the target syntactic structure. Such an extreme problem is not likely 
in a careful study such as that of Dormuth et al. More generally, though, 
semantic constraints on the answer to an item's question that result from 
choice of content words for the item's paragraph are difficult to deter- 
mine and may not be completely controlled in studies such as that of 
Bormuth et al. and the present experiment. Again, this may be a moot 
point if such constraints are perfectly correlated with syntax, but the 
present results rule out this possibility. 

In addition to semantic difficulty and the extent to which the para- 
graph "gives away" the answer, there is a third potential source of var- 
iance in the difficulty ordering produced by the Bormuth et al. method. 
Two passages that have a target syntactic structure in common may dif- 
fer in the extent to which they can otherwise be processed to the point 
at which the critical structure is relevant. One may sensibly hypothe- 
size, for example, that an anaphora cannot be comprehended unless both 
it and its antecedent are simultaneously in operating (short-term) mem- 
ory. For example, Bormuth et al. found that items like (1) are compre- 
hended 87 percent of the time, while items like (2) are comprehended 
only 66 percent of the time, by fourth-graders. If he had used a differ- 
ent pro-clause item like (3), instead of (1), perhaps only half the children 
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would have understood pro-clause fornis, thus reversing the ranking 
of clause denionstratives and pro-clauses (so). Passage 3 is longer 
than (1) and niay be niore difficult to encode due to its idiomatic content. 
Most important, so^ stands in place of a much more complex construc- 
tion in (3) than in (1). Unless a nine-year-old child is able to record 
splice the mainbrace into a single image, he may lack the ability to pro- 
cess that phrase to the point of being able to resolve the anaphoric refer- 
once. No control procedure is available to insure that Bormuth's rank- 
ing arises only from syntax differences and not from differences in pas - 
sage wording or the amount of processing required to get from surface 
to underlying structure. 

(1) Joe may cry. If _so , the rest of us will be sad. 

(2) Joe is coming home. That is the best news I've 
heard all week, 

(3) Joe may splice the mainbrace, If^i the rest 
of us will be glad. 

The present results arise from what was expected to be a screen- 
ing task for an experiment on memory for syntax. Thus, they provide 
information about only a subset of nine out of Bormuth et al, 's fourteen 
anaphora forms. The differences between the two studies are: (1) the 
present study used oral, constructed responses while Bormuth et ai. 
used written, multiple-choices responses; (2) the present study explicitly 
controlled the number of semantically plausible potential answers in each 
passage; and (3) the location of the target structure in the passage was 
counterbalanced in the present study. 
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Method 



Subjects . Forty students from a campus laboratory school and 

40 students from an urban public school participated as subjects. The 

campus ^roup ranged in age from 8.0 to 10.0 years with a median of 

S. 8, while the urban group ranged from 8.0 to 11.5 years with a median 

3 

of 9.4. All were in third or fourth grade. 

Materials . Fourteen anaphora forms were tested in this study. 
They are listed with examples, in Table 1. Three items were written 
for each form. An item consisted of a paragraph plus a question. The 
paragraph, in turn, consisted of two filler sentences plus a two-sentence 
critical structure sequence. The first of the two critical sentences con- 
tained an antecedent which was referenced by an anaphora in the second. 
Each of the three items for a form had a different location for the criti- 
cal sentences in the paragraph: before, between, or after the filler sen- 
tences. Each item's passage was constructed so that there were two 
semantically sensible answers to the question. The correct choice was 
determined by the anaphora syntax. The question for each item was 
written by substituting the appropriate wh-word for the anaphora and then 
applying the shortest sequence of transformations that would turn the ana- 
phora sentence into a question. 

Each question was typed on a 21.6 x 27.9 cm sheet in 0 . 42 cm 
gothic type and covered with a clear plastic sleeve. The 42 pages were 
presented to subjects in a loose -leaf binder. Order of occurrence of the 
items was approximately counterbalanced. 



The campus laboratory school actually had two-year groupings, 
one of which was equivalent to third and fourth grades. 
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Procedure , The procedure was first explained to S. Each S 
worked individually and at his own pace reading each paragraph and then 
orally answering the question. The Ss fronn each school were split into 
two groups of 20 each. One group read the passages (generally silently), 
while the other group followed along reading (silently) while they heard 
a tape recording of the passage as well. After completing 30 of the 42 
passages, each S stopped for a brief cookie break. Occasionally, S 
would respond to a question by stating the anaphora word (e.g. , he). 
The subject was then asked to be more specific (e. g. , Whom do you 
mean by he ? ). 

Scoring . Each response to a passage by a given S was punched 
onto a separate computer card. A list was then produced, sorted by 
item. This made it possible to examine all answers to an item at once. 
Each answer was scored as correct only if the verbatim antecedent of 
the anaphora in the given passage was stated or implied by a stated syn- 
onym of superordinate category term. Answers referring to more en- 
tities besides the correct antecedent were also counted wrong. Scoring 
was verified by a second observer. 

Results 

Two reliability measures were computed. The cell means for 
the fourteen anaphora types were compared for the two schools (r - 
0.94, 12, p < .001) and for the two presentation conditions (r - 

0.88, = 12, 2 <.001). Both correlations are quite high, suggesting 
that these results are relatively stable for the particular items used. 
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To determine whether the anaphora forms differ in their diffi- 
culty, a 20 (Subjects) X 14 (Anaphora) X 2 (Schools) X 2 (Presentation 
niodc) partially nested analysis of variance was performed on the scores 
(zero to three correct) of Ss on each of the anaphora forms. There were 
significant differences among the anaphora forms (F - 19.6, df = 13/988, 
p • .0001) and no interactions of Anaphora Vvith-Sc>ix)ols or Presentation 
mode (F^s <^ 1. 28, df = 13/988). The percentage correct for each of the 
anaphora types is shown in Table 1. 

Schools was a significant variable (F = 27.5, df = 1/76, p< .0001), 
with the campus lab school (M = 2.42) showing higher performance than 
the urban public school (M = 2.04), There is also some reason to sus- 
pect that while the urban school Ss performed equally under the two pre- 
sentation conditions (Ms = 2.03, 2.04), the campus lab school Ss bene- 
fited from the read-and-hear condition {Ms = 2. 54, 2. 29; F. = 2. 89, df = 
2/76, 2 < • 10)- 

The important result is the comparison of the rank orders of dif- 
ficulty for those nine anaphora conditions common to both the present 
study and that of Bormuth et al. (1970). The mean percent correct for 
each anaphora form in each study can be seen in Table 1. There is a 
significant negative correlation between the two sets of means (Spear- 
man rho = 0. 66, N = 9, £ < . 05)'. 

Discussion 

Since Bormuth et al. (1970) suggest that their difficulty ordering 
may be the basis for a learning hierarchy for anaphora syntax, it is im- 
portant to be certain that their data and the present data cannot possibly 



8 



be a basis for the same hierarchy. The negativ^e rank correlation gives 
sonic evidence in this regard. However, it would be possible to find 
such a correlation if one subject sample possessed the skills needed to 
comprehend Structure A, while a second subject sample had not yet ac- 
quired those skills. If both samples performed at a middle level on 
Structure B, then the first would display a rank ordering of A > B, whiie 
the second i ad an ordering B ^ A. Since many structures have a large 
set of interpretation rules, the relative order of difficulty could change 
as rules with differing; importance are acquired. 

This is not the case for the present data since soirie forms, such 
as the two pro-verb forms and the pro-clause (so) form, are better com- 
prehended by the Bormuth et al. Ss, while others, such as personal pro- 
noun, are better comprehended by the present S^s. This complete cross- 
over requires us to reject any argument that the differences between re- 
sults of the Bormuth procedure and the present study are due to develop- 
mental differences in the two studies' subject pools. 

The present study was intended as a screening task for an experi- 
ment derived from Bormuth' s work. Thus, every effort was made to 
reproduce any potentially critical procedures of Bormuth's. The results 
obtained are statistically reliable. We cannot help but conclude that syn- 
tax is not the basis of a hierarchy of comprehension skills that have not 
been completely acquired by the time children are in fourth grade. Any 
structure for which high performance is shown in either the present 
study or that of Bormuth et al. must be assumed not to present a straight- 
forward syntactic hurdle to fourth-grade children. 

Why, then, do children perform poorly on a particular structure 
on even one of the two tests. There are two potential reasons: (I) the 
child may not know the interpretation rules required to understand the 



ERIC 



9 



structure in a particular semantic context; or (2) he may lack the real- 
time capacity for applying those rules. There is a good reason to be- 
lieve that both of these factors play a role but that the second is of pre- 
dominant importance for childien as old as the present Ss. The avail- 
able evidence is primarily restricted to studies of pronominalization. 

Fredrick, Golub, and Johnson (1970) have found that multiple- 
choice pronoun items similar to those of Bormuth et al, are correctly 
answered only 28 to 50 percent of the time. The items are grammati- 
cally similar to the personal and relative pronoun items of the present 
study, which are correctly answered 92 and 78 percent of the time, re- 
spectively. The apparent difference is in the level of semantic ambiguity 
in the two cases. Consider the following examples, from Fredrick et 
al. and the present study, respectively; 

(1) The notebook on her desk covered up my draw- 
ing which was very messy. [Identify which . ] 

/a. notebook 
^ b. desk 

c. covered up 

d. drawing 

e. messy 

(2) Two men were walking down the street. One 
man had on a hat. The big man who was stand- 
ing on the corner is my father. Who was stand- 
ing on the corner? 

The first example is oemantically more complicated. It requires 
that the child realize that the "closest semantically acceptable ante- 
cedent" rule applies. There are three semantically acceptable potential 
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antecedents, while there are only two potential answers to (2). Even 
children who get (1) and (2) correct will likely waver in handling (3). 

(3) I put the package on Ihe table. Because it was 
tilted, it fell off . [Identify it. ] 

a. package 

b. table 

c. tilted 

d. fell 

e. I 

The point is that grammatical rules for nine-year-old children 
are not abstract structures that apply mechanically. They are inextri- 
cably bound up with semantics (cf. Palermo & Molfese, 1972). The pres- 
ence of a subset of the pronoun interpretation rules in even five-year- 
olds (Chomsky, 1969) does not mean that these rules operate abstractly 
and free of semantic influence. The potential success of syntax tests 
with uncontrolled semantics is low in general; it is practically non-exis- 
tent for pre-formal-operations children. 

The second potential reason for variability in tests of syntactic 

competence is differences in the extent to which a given item exceeds 

or stays within the child's channel capacity as an information processor. 

4 

Recent findings by the present author suggest that adult-like compre- 
hension of personal pronoun sentences is more likely when a child can 
replace cumbersome surface-structure segments with imaginal codes. 
Thus, imagery factors apparently play a role in comprehension perfor- 
mance, also. 

^Lesgold, A. M. Memory structure for pronoun sentences: 
Imagery effects on comprehension channel capacity, in preparation. 
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These arguments suggest that the design of a syntax comprehen- 
sion curriculum cannot readily be based upon tests of the sort described 
here and by Bormuth et al. Such tests have uncontrolled variance due 
to in-iagory and semantic factors. The structures tested are often (but 
not always, cf. Palermo & Molfese, 1972) understood by children when 
presented in sufficiently semantically simple circumstances. On the 
other hand, sufficiently complex semantics will probably override knowl- 
edge of any structure in young children. Perhaps semantic analysis, 
at tlie level of care that Bormuth (1970) has brought to syntactic analysis, 
will be useful in the design of instruction. However, at the present, the 
use of difficulty orderings for syntax without regard to semantics is not 
likely to lead to improved instruction in comprehension. 

It is certainly possible that abstract syntax rules are being ac- 
quired during the ages from which Bormuth' s and the present samples 
were drawn. Further, it is conceivable that one might want to test for 
the presence of such rules. This could be done by using nonsense words 
instead of lexical words in the various item forms used in this study. 
This would yield measures of "pure" syntax ability. However, many 
syntactic rules interact with semantics, so this "pure" ability might not 
be very relevant. 
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